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Abstract 


Stationary  points  for  a system  with  a convex  polyhedral  set 


and  a continuously  differentiable  function  with  positive  definite 
derivatives  are  computed  by  iteratively  solving  the  linearized  pro- 
blem. The  procedure  is  shown  to  be  a mixing  of  a finite  number  of 
Newton  methods  and  to  have  a local  convergence  rate  which  is  quad- 
ratic. The  stationary  point  problem  is  of  the  type  arising  from 
the  PIES  energy  model. 
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A LOCALLY  QUADRATICALLY  CONVERGENT 
ALGORITHM  FOR  COMPUTING  STATIONARY  POINTS 

I j 

1.  Introduction 

Let  Y and  Z be  two  sets  in  Rn  and  let  g : Z -*■  Rn  be 
a function.  A point  yA  in  Y and  in  the  domain  of  g is  defined 
to  be  a stationary  point  or  solution  to  the  variational  problem 
(Y,g)  if 


(y*  - y)  e(y*)  i o 


for  all  y in  Y. 

In  this  paper  we  are  concerned  with  the  computation  of 
stationary  points  of  (X,(e,f))  ■ (X,e,f)  where  X in  Ri+m  **  Rn 
is  of  form  {(u,v)  : Au  + Bv  £ C}  and  (e,f)(u,v)  ■ (e,f(v))  . The 
vector  e is  any  element  of  R*\  Let  V be  the  set  of  v for  which 
there  is  a (u,v)  in  X.  The  function  f : V -*■  R™  is  assumed  to  be 
continuously  differentiable  (C^)  and  to  have  derivatives  that  are 
positive  definite  (p.d.)  but  not  necessarily  symetric. 

The  algorithm  defined  below  is  shown  to  be  a mixing  of  a finite 
number  of  Newton  methods.  If  v^  is  near  vA  where  (uA,vA)  is  a 
stationary  point  of  (X,e,f),  then  the  iterates  v^  generated  by  the 
algorithm  are  shown  to  converge  at  a superlinear  rate  to  v*.  If,  in 
addition,  the  derivative  f'  of  f is  Llpschitz  continuous  at  v^, 
the  convergence  rate  is  quadratic. 

I 


1 


For  convenience  we  shall  refer  to  v as  a v-stationary 
point  of  (X,e,f),  if  there  is  a u such  that  (u,v)  is  a stationary 
point  of  (X,e,f).  Also  for  each  v we  define  the  function  fv  : Rm  -*■  R™ 
by 


fv(v)  - f (v)  + f ' (v)  (v-v) 

Regard  fv(v)  as  an  approximation  of  f(v)  from  v . 

Define  G : V -*■  V by  setting  G(v)  to  be  the  v-stationary  point 
of  (X,e,fv)  . The  algorithm  is  defined  by  vfc  - G (vQ)  for  k ■ 1,  2,  ... 
where  Vq  is  selected  arbitrarily  in  V.  That  is  to  say,  the  kth 
iterate  is  v^  » G(v^  where  v^_^  is  the  (k-l)th  iterate. 

Executing  step  k,  that  is,  computing  is  a “atter  of 

solving  a linear  complementary  problem.  Procedures  for  solving  such  a 
problem  can  be  found  in  Cottle  [ 2]  and  Lemke  [ 9 ],  however,  of  special 
Interest  here  is  [3].  The  effort  required  to  evaluate  G(vk>  is 
approximately  that  of  solving  or  post-optimizing  one  linear  program 
with  the  simplex  method  where  the  matrix,  including  the  basis,  is 
n'  + n by  2(n'  + n)  and  (A,B)  is  n’  by  n . 

The  motivation  for  this  paper  was  due  in  part  from  an  interest 
in  the  PIES  energy  model  and  algorithms  thereof , see  Hogan  [ 5 , 6 ] • 

Let  us  briefly  describe  this  model  and  how  it  can  be  formulated  as  a 
stationary  point  problem.  First,  on  the  demand  side  of  the  model  we 
have  a function  3 of  prices  p.  At  the  vector  of  prices  p there 
is  a vector  of  demands  i?(p) ; it  is  assumed  that  2 is  and  its 
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derivatives  are  negative  definite  but  not  necessarily  symetric.  On 
the  supply  side  we  have  the  linear  program 


minimize : cx 
x 

subject  to:  Ax  * a 
Bx  - q 


x > 0 


where  q is  the  vector  of  quantities  to  be  produced.  Roughly,  an 
equilibrium  of  the  model  is  defined  to  be  a price  p*  such  that  the 
marginal  cost  of  producing  at  levels  ^(p*)  is  p*  . More  precisely, 
if  the  linear  program  is  solved  for  q * ^?(p*) , then  pfc  must  be  a 
set  of  optimal  dual  prices  for  q . In  other  words,  we  desire  a p* 
for  which  there  is  a pair  (**,P*)  which  solves  the  linear  program  (1) . 


maximize:  Xa  + p j?(p*) 

U»P) 

subject  to:  XA  + pB  c 


By  setting  e * - a and  f ■ - Q it  is  clear  that  this  is  the  stationary 
point  problem  where  f has  p.d.  derivatives.  If  the  Inverse  demand  . 
function  Sf  - 2~^  is  available  we  can  state  the  problem  as  that  of 
finding  a q*  for  which  there  is  a (x*,q*)  solving  the  linear  program  (2). 


minimize:  cx  - ^(q*)  q 

x.q 

subject  to:  Ax  * a 

Bx  - q ■ 0 


x > 0 
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Setting  e ■ c and  f ■ - 9*  we  again  have  a stationary  point  problem 
with  p.d.  derivatives  for  f . It  is  easy  to  show  that  p*  solves 
(1)  if  and  only  if  q*  solves  (2)  where  q*  - ^(P*)  • 

The  PIES  algorithm  proceeds  in  an  effort  to  find  q^  by  solving 
at  the  (k+l)th  iteration  the  linear  program 

minimize: 

CX  ~ C<qk,q) 

x.q 

subject  to:  Ax  » a 

Bx  - q » 0 x j>  0 


for  w^ere  ^(q^.q)  - as  a function  of  q is  a convex  piecewise 

linear  function  which  approximates  the  integral 

h q1 

I / (z;qv)  dz 

i-l  0 

where 

/ \ 1 i-l  i+1  h. 

•••»  <1^  » z»  •••» 

and  q1  is  the  ith  component  of  q . 

The  algorithm  proposed  here  for  computing  q^  would  proceed  at 

k+1 

the  (k+l)th  iteration  by  solving  for  a q such  that  (\+1»  qk+1) 
solves  the  linear  program 
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minimize:  cx  - q 

x,q 

subject  to:  Ax  - a 

Bx  ■ q x>0 

Clearly  the  subproblems  of  the  PIES  algorithm  are  considerably  easier  to 
solve  than  those  of  our  algorithms;  roughly,  we  are  comparing  an'  by 
n + n'  linear  program  with  a (n'+n)  by  2(n'+n)  linear  program.  In 
both  cases  one  can  use  the  solution  of  the  last  iteration  to  initiate 
the  current  iteration. 

The  PIES  algorithm  is  reported  to  perform  remarkably  well  [5,6]. 
However,  on  a theoretical  level  it  has  to  date  evaded  a complete  analysis. 
A number  of  pertinent  investigations  include  Ahn  [ 1 ] , Irwin  [ 7 ] , 
Thrasher  [22],  and  Tse  [24].  In  particular,  Ahn  has  demonstrated  that 
for  a linear  demand  function  the  convergence  is  global  with  a 
linear  rate  of  convergence. 

Looking  now  in  other  quarters  of  the  literature  we  first  note  the 
work  of  Mathiesen  [10,11;12].  He  observed  that  problems  of  type  (1)  and 
(2)  with  linear  2 or  & could  be  cast  into  the  form  of  the  linear 
complementarity  problem  and  solved,  if  there  is  a solution,  by  Lemke's 
method.  Solving  such  a problem  is  step  k of  the  algorithm  of  this 
paper,  as  already  mentioned. 

Next  we  observe  that  the  algorithm  of  this  paper  is  closely 
related  to  the  scheme  of  Wilson  [25]  for  maximizing  a nonlinear  objective 
subject  to  nonlinear  constraints;  his  scheme  proceeds  by  solving  a 
sequence  of  quadratic  programs.  Wilson  did  not  have  a proof,  however. 
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in  recent  years  there  have  been  a number  of  arguments  validating 
this  type  of  algorithm  wherein  the  schemes  are  shown  to  be  like 
Newton's  method  in  the  tail,  see,  for  example,  Han  [4],  Palomares 
and  Mangasarian  [14],  Powell  [15,16],  Tapia  [20,21]  and  Robinson  [17,18]. 
Often  these  schemes  utilize  only  the  derivatives  of  the  objective  and 
not  the  objective  itself.  Hence,  these  schemes  could  be  used  as 
well  for  stationary  point  problems.  Following  this  reasoning,  one 
obtains  after  suitable  specialization  and  relaxation  the  scheme  of 
this  paper.  Our  analysis  here  does  not  use  multipliers  and  we  have 
been  able  to  drop  the  regularity  conditions  required  of  these  procedures. 

The  studies  [ 4 ,14,15,16,17,18,20,21]  focus  on  the  fact  that 
superlinear  convergence  is  retained,  if  the  quadratic  form  at  each 
Iteration  only  approximates  the  Hessian  (our  derivative) . The 
* essential  idea  is  that  the  computation  of  the  Hessian  at  each  iteration 
is  too  expensive  and  that  an  approximation  serves  almost  as  well.  No 
doubt  the  algorithm  of  the  paper  could  be  improved  in  this  sense.  Per- 
haps the  PIES  algorithm  is  related  to  the  algorithm  of  the  paper  in  this 
regard . 

Also  of  interest  are  the  global  schemes  of  Rockafellar  [19]  and 
Todd  [23]  for  solving  the  stationary  point  problem;  however,  both 
schemes  are  quite  different  than  what  is  contemplated  here.  One  could 
invlsion  using  the  algorithm  proposed  here  as  a tail  routine  for  these 
global  procedures. 
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2.  Newton's  Method 


Let  us  consider  the  application  of  Newton's  method  to  solve 
the  system 

(3)  H(u,v)  - Eu  + F(v)  ■ 0 

of  n equations  in  n variables  (u,v)  where  Eu  is  linear  in 
u in  R*  and  F is  C*  with  domain  V . 

j, 

Given  the  estimate  (u0»v0^  in  R x V the  Newton  iterate  is 
any  (u^.v^)  in  whicl»  solves 

Euq  + F(vq)  + ECu^Uq)  + F’ (vQ)  (v^Vq)  = 0 

that  is,  which  solves 

(4)  Eu^  + Fv^Vj)  - 0 

Note  that  (u^,v^)  depends  upon  v^  but  not  Ug  and  that  such  a 
(u^,v^)  may  not  exist.  Let  us  assume  that  (u*tv*)  solves  (3)  and 
that  any  solution  (u,v)  to 

Eu  + Fv^(v)  - 0 


has  v ■ vfc  . Finally,  let  us  consider  only  those  Vg  for  which 


v falls  in  V.  We  argue  that  Newton's  method  in  this  setting  has  the 

following  convergence  properties.  First,  there  is  a 6 > 0 and  8 < 1 

such  that  if  lv^  - v^l  <_  6 then  Iv^  - v^B  <_  bIv^  - v^B  . Second,  0 

can  be  made  arbitrarily  small  by  making  6 sufficiently  small.  Third,  if 

if  F'  is  Lipschitz  continuous,  then  there  is  a y so  that 

2 

Bv^  - v^l  <_  yBvq  - v^B  for  sufficiently  small  6 . 

Observe  we  do  not  make  the  customary  assumption  that  the  matrix 
(EjF'Cv^))  is  nonsingular,  however,  we  have  required  that  F'Cv^)  be 
of  full  column  rank  through  the  uniqueness  condition. 

Let  d^,  . . . d^  be  an  orthogonal  basis  for  the  null  space 
{u  : Eu  ■ 0}  of  E . Let  D be  the  matrix 


Then  for  any  u there  exist  a unique  u such  that 


Eu  “ Eu 


Du  - 0 


For  any  matrix  M let  M denote  the  submatrix  of  rows  in- 

a 


dexed  by  a . Select  a so  that  the  matrix 
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is  nonsingular.  Consider  the  system  (5). 


Of  course,  (u^,v^)  is  a solution  to  (5).  Now  we  apply  Newton's 
method  to  the  system  (5)  at  (u^.v^)  . Given  in  V the  system 


is  solved  for  (u^,  v^)  . If  we  assume  that  v^  lies  in  V,  the 
usual  argument  for  the  convergence  of  Newton's  method  now  obtains, 
namely,  a nonsingular  derivative  at  the  solution  and  continuous 
differentiability,  see  Ortega  [13],  for  example.  Since  the  computa- 
tion of  (u^,v^)  does  not  involve  u^  we  may  assume  that  u^  “ u^ 
and  conclude:  First,  there  is  a 5 > 0 and  B < 1 such  that  if 
l(u*,vQ)  - (u^.v^)!  <6  then  B^.v^  - (u*,v*)l  ^bI^.Vq)  - (uA,v*)l 
Second,  6 can  be  made  arbitrarily  small  by  making  5 sufficiently 
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. Uniqueness  and  Existence 


In  this  section  we  cite  a number  of  uniqueness  and  existence 
results  which  are  necessary  for  or  clarify  the  subsequent  develop- 
ment. We  shall  use  the  box  norm,  that  is,  Hxll  « maxlx^l  . 

The  results  of  this  section  are  about  the  structures  X and 
X in  RZ+m,  V - {v  : (u,v)  e X} , e e Rl , f : V -*■  Rm,  and 

g : X -►  R4+m  . If  X is  of  form  {(u,v)  : Au  + Bv  <_  c},  then  we 

shall  refer  to  X as  a cell. 

We  define  f to  be  strictly  monotone  if 

(v  - v)  (f  (v)  - f (v) ) 

is  positive  for  all  distinct  v and  v in  V,  and  we  define  f to  be 
strongly  monotone  if  for  some  a > 0 

(v  - v)(f(v)  - f(v))  >_  a Bv  - vll^ 

for  all  v and  v in  V . We  say  the  derivatives  of  f are  p.d.  if 

y f'(v)y  is  positive  for  all  (y,v)  in  R™  x V with  y j*  0,  and  we 

say  the  derivatives  of  f are  uniformly  p.d.  if  for  some  a > 0 

y f ' (v)y  £ a ly I ^ 

for  all  (y,v)  in  Rm  x V . 
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and  hence  vQ  ■ v*  . 


For  v in  V define  U(v)  to  be  the  set  of  all  optimal  solu- 
tions u of  the  program 


minimize:  eu 


subject  to:  (u,v)  e X 


Lemma  3:  If  f is  strictly  monotone  and  is  the  v-stationary 

point,  then  the  set  of  stationary  points  of  (X,e,f)  is 

U(v*)  x {v*}  . 0 


Lemma  4:  If  f is  strictly  monotone  and  some  stationary  point  of 
(X,e,f)  is  also  a stationary  point  of  (X,e,f),  then  all  stationary 
points  of  (X,e,f ) are  stationary  points  of  (X,e,f)  . 


Proof:  If  and  (u2»vi)  are  both  stationary  points  of 

(X,e,f)  then  eu2  - e^  and  hence,  (u^Vj.)  is  a stationary  point 
of  (X,e,f ) . 0 

Lemma  5:  If  (u,v)  is  a stationary  point  of  (X,g)  and  (u,v)  is 
in  X,  then  (u,v)  is  a stationary  point  of  (X  n X,g)  . S 

For  X closed  and  convex  define  it  : Rn  -*■  X to  be  the  pro- 
jection to  X,  namely,  ir(x)  is  the  point  in  X closest  to  x; 
observe  that  n is  continuous. 
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Lemma  6:  If  X is  closed  and  convex,  then  x is  a stationary  point 
of  (X,g)  if  and  only  if  ir(x  - g(x))  - x . H 

Lemma  7:  Let  X be  convex.  If  x is  a stationary  point  of 
(X  n X,g)  and  x is  interior  to  X,  then  x is  a stationary  point 
of  (X.g)  . 

Proof:  x is  the  point  in  XflX  closest  to  (x  - g(x)).  Thus, 
is  the  point  in  X closest  to  (x  - g(x))  . 

By  a face  <p  of  a convex  set  X we  mean  the  meet  of  X 
with  a supporting  hyperplane  or  the  set  X itself. 

Lemma  8:  If  X is  convex,  x is  a stationary  point  of  (X,g), 
x lies  in  the  relative  interior  of  a face  (p  of  X,  then  x is 
stationary  point  of  (X,g)  where  X is  the  affine  hull  of  (p . 

Proof : If  x is  in  X,  then  x - x + X(y  - x)  where  y is  in 

consequently,  (x  - x)  g(x)  ■ X(x  - y)  g(x)  <_  0 . 

Lemma  9:  If  X is  a cell  and  (X,e,f)  has  a stationary  point,  then 

Au  0 eu  < 0 

has  no  solution. 


0 


and 
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Proof:  It  follows  immediately  from  the  definitive  of  a stationary 
point.  0 


Clearly  Au  <_  0 with  eu  < 0 has  no  solution,  if  and  only  if 
for  each  sequence  (u^.v^)  in  X for  which  eu^  tends  to  then 

vfc  tends  to  «,  that  is,  tends  to  +»  . Obviously,  the  PIES 

models  (1)  and  (2)  possess  this  property  or  no  solution  would  exist; 
in  any  case,  one  can  give  a simple  economic  argument  that  such  should 

K 

be  the  case. 

The  next  lemma  guarantees  that  each  step  of  our  algorithm, 
namely,  the  evaluation  of  G(vk>,  can  be  executed.  Let  X » {(u,v)  : 
Au  + Bv  < c}  . 


Lemma  10:  If  X is  a cell,  Au  <_  0 with  eu  < 0 has  no  solution, 
and  f'(v)  is  p.d.,  then  (X,e,fv)  has  a stationary  point. 


Proof:  According  to  [3]  if  Lemke's  algorithm  cannot  compute  a 
stationary  point  of  (X,e,fv)  then  there  is  a ray  {(u^,v^) 

+ eCu^.Vj)  : 0 >_  0}  in  X with  u2  e + v^  fv^  + 0v2>  negative, 
that  is,  u2  e + v2  f(v)  + + 0v2f'(v)v2  negative  for  all  0 . 

Since  f'(v)  is  p.d.,  v2  ■ 0 . Hence,  A u2  <_  0 and  u2  e < 0 
which  is  contrary  to  our  hypotheses.  0 


Lemma  11:  If  X is  a cell  and  Au  < 0 with  eu  < 0 has  no 
solution,  then  there  are  numbers  n and  £ for  which  eu  n Ivl  + £ 
for  all  (u,v)  in  X . 
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Proof ; Consider  the  set  (u,v,w,z)  such  that  (u,v)  Is  In  X, 
v ■ eu,  z _>  v^,  and  z _>  - for  all  1 . Use  Dines-Fourier-Motzkin 
elimination  to  reduce  the  system  to  inequalities  in  two  variables 
w and  z . Among  these  inequalities  there  must  be  at  least  one  of 
form  w ^ nz  + £ . 0 

The  next  result  can  be  found  in  Karamardian  [ 8 ] . 

Lemma  12:  If  X is  convex  and  compact  and  g : X -*■  Rn  is  con- 
tinuous, then  (X,g)  has  a stationary  point. 

Proof:  it(x  - g(x))  is  a continuous  map  from  X to  X and  has  a 
fixed  point.  IE) 

Lemma  13:  If  X is  closed  and  convex,  there  are  numbers  n and  £ 
for  which  eu  ^ n Jvl  + £ over  X,  and  f is  strongly  monotone,  then 
(X,e,f)  has  a stationary  point. 


Proof:  Let  XL 

be  the  set 

of  x 

in  X such  that  Ixl  <_  t 

for 

t - 1,  2,  3,  ... 

. Select 

(u,v> 

in  some  Xf  . For  t r 

let 

(ut,vt)  be  a stationary  point  of  (Xt,e,f)  . If  the  vt's  have  a 

cluster  point  v as  t -*■  + «,  then  v*  is  a v-stationary  point 

* 

of  (X,e,f)  . Thus,  suppose  that  v -*■  « as  t -*•  + «•  . We  have 
(ufc  - u)  e + (vt  - v)  f (vt)  < 0 
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4.  Continuity  of  G 


Given  V ■ (v  : 3u,  Au  + Bv  £ C}  and  f : V -*•  Rm  which  is 

with  p.d.  derivatives,  we  define  G : V -*■  V by  setting  G(v)  to 

be  the  v-stationary  point  of  (X,e,fv)  . Our  algorithm  generates 
the  sequence  G (v^)  . In  this  section  we  show  G to  be  continuous. 

The  next  lemma  is  a standard  result.  Let  g and  h be 
functions  from  Y to  Y where  Y is  in  Rn  . By  Ig  - hi  we 
mean  the  supremum  of  lg(y)  - h(y)l  for  all  y in  Y . 

Lemma  14;  If  g is  continuous,  Y is  compact,  and  g has  fixed 
points  Y^,  then  for  every  e > 0 there  is  a 5 > 0 such  that 
Ig  - hi  <,6  implies  that  all  fixed  points  of  h lie  within  e of  Y^  . 

Proof : Let  26  equal  the  min  of  lg(y)  - yl  over  the  set  {y  e Y : 

ly  - Y I > e}.  For  ly  - Y I > e we  have  lg(y)  - yl  > 26  . So, 

q — q — 

lh(y)  - yl  > lg(y)  - yl  - lg(y)  - h(y)l  ^26-6*6>0.  0 

We  define  the  distance  d(S^,S2>  between  two  sets  and 

in  Rn  to  be  (the  Hausdorff  metric) : 

max  min  lx  - yl  + max  min  lx  - yl 
x^S1  yes2  yes2  x^ 

R1+m  and  f : V Rm  be  C1  with  p.d. 
v-stationary  point  of  (X,e,fv)  , if  it 


Lemma  15:  Let  X be  a cell  in 
derivatives.  Let  G(v)  be  the 
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r — 1 


exists.  If  G is  well  defined  for  any  v,  then  it  is  well  defined 
on  V and  is  continuous. 

Proof ; If  (u^.v^)  a stationary  point  of  (X,e,fVg),  then 
eu  < 0 with  Au  ^ 0 has  no  solution,  so  G is  well-defined  by  Lemma  10. 
Select  (A,B,c)  so  that  X ■ { (u,v)  s X : Au  + Bv  < c}  is  bounded 
and  Au  + Bv  < c for  all  (u,v)  with  !(u,v)  - (u^,v^)l  <_  3e  . 

Select  <_  e so  that  lv  - v^l  £ implies  d(U(v),  U(v^))  <_  e 

A 

where  U(v)  is  the  set  u that  minimizes  eu  subject  to  (u,v)  in 

A . A 

X . Using  Lemma  14  for  X and  the  function  g(u,v)  » ir((u,v)  - 
(e,fvQ(v)))  where  ir  is  the  projection  to  X,  select  6 so 
that  if  #Vq  - v^I  <_  <S,  then  each  fixed  point,  say  (u^.v^),  of 

/V 

*((u,v)  - (e.fv^tv)))  is  within  of  some  fixed  point  of 

fi((u,v)  - (e,fvQ<v))),  say  • Since  lv3  - v^l  <_  there  is 

A _ _ 

a u^  in  U(v3>  so  that  lu^  - u^I  <_  e ; (u^.v^)  is  a stationary 
point  of  (X,e,fv2)  . But  l(u3>v3)  - <.  2e  so  (u3,v3>  is 

a stationary  point  of  (X.e.fv^)  and  lv3  - v^I  <_  e . 0 

Thus,  if  G is  defined  anywhere,  it  is  defined  everywhere  and 
is  continuous. 
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5.  The  Functions  G and  Mixing 

Let  <p^  be  any  closed  face  of  X of  any  dimension  and  let 
XA  be  the  affine  hull  of  (p±  . For  v in  V define  G^v)  to  be 

the  v-stationary  point  of  (X±,e,fv),  if  it  exists;  recall  that  if 
(X^.e.fv)  has  a v-stationary  point,  then  it  is  unique.  We  proceed 
to  show  that  the  action  of  G^  is  equivalent  to  that  of  Newton's 
method  as  in  Section  2. 

Define  H±  : R*"  * V -*•  Rn  by  H^u.v)  - ^((u.v)  - (e,f(v))) 

- (u,v)  where  is  the  projection  of  X^  . ir^  is  an  affine 

function  and  can  be  expressed  in  the  form  x^(x)  **  Px  + p . Hence, 

H is  linear  in  u . Given  (u0,vQ)  in  X the  Newton  iterate  is 
any  solution  (u^,v^)  co 


H<VV  + H?(uo'vo)((ui,vi)  " (Vvo»  " ° 


or 


P((u0,vQ)  - (e,f (vQ)))  + p - (u0,v0) 


+ (P  - P( 


f'Jv  )>  ‘ - (uQ,v0))  - 0 or 


P((u1,v1)  - (e,fvft (v, )) ) + p - (u^v,)  - 0 


'0W1 


11 


or 


i»l<(ul*vl)  - (e,fVg (v^)))  - (uj.Vj)  “ 0 • 


' * 


That  is  to  say,  given  vQ  in  V,  the  Newton  iterate  is  any  (u^v^ 
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which  is  a stationary  point  of  (X^,e,fVg)  . That  is  to  say,  the 
portion  of  the  Newton  iterate  is  g^(vq)  * 

l 

Now  suppose  that  (u^.v^)  in  R * V is  a stationary  point 
of  (Xj,,e,f)  . The  v-stationary  point  of  (X^.e.fv^),  namely  v^, 
is  unique.  We  can  now  apply  the  results  of  Section  2;  we  consider 
only  the  for  which  G^(Vg)  is  in  V . 

First,  there  is  a 6 > 0 and  6 < 1 such  that  if  llv^  - v^l  5 
then  Ig^v^)  ” v**  — ~ v*^  * Second,  B can  be  made  arbitrarily 

snail  by  making  5 sufficiently  small.  Third,  if  f'  is  Lipschitz 
continuous,  then  there  is  a y and  6 > 0 so  that  Ig^(Vq)  - <_ 

*,vo  ~ v*'2  for  ,v0  ~ v*!  6 • 

Given  Vq  let  (u^.v^)  be  a stationary  point  of  (X,e,fvQ)  . 

Then  (u^,v^)  is  Interior  to  some  face  (f>^  of  X . We  then  see  that 

«V  - 0l(v0)  - vt 

which  is  to  say  that  G is  a mixing  of  the  G^'s  or  that  G is  a 
nixing  of  a finite  number  of  Newton  methods. 


6.  Local  Convergence  Rate  of  G 


Let  be  the  v-stationary  point  of  (X,e,f)  . We  show 

lc 

that  if  Vq  is  sufficiently  near  v*,  then  G (Vq)  tends  to 
at  a super linear  rate,  and  if,  in  addition,  f'  is  Lipschitz  con- 
tinuous, the  rate  is  quadratic.  Regard  the  vectors  d and  d*  in 
Rn  as  constant  functions  from  Rn  to  Rn  . 

Lemma  16:  Assume  xA  is  a stationary  point  of  (X,dA)  . There 

is  an  e > 0 such  that  if  Id  - d^H  <_  e and  x is  a stationary 

\ 

point  of  (X,d) , then  x is  in  the  face  {y^X  : y d*  ■ x*  d^}  of  X . 


Proof:  Suppose  the  contrary  and  we  have  d^  -*•  d*  and  in  X so 

that  d^  x*  < d^  x^  and  d^  x^  < dfc  x for  all  x in  X . Select 
an  Infinite  subsequence  so  that  all  x^  lie  in  the  relative  interior 
of  one  face  of  X . Letting  x be  any  element  in  the  relative 
interior  of  this  face  we  have  dA  x*  < d*  x and  d^  x <_  d^  x for 
all  x in  X . Setting  x - and  taking  the  limit  we  get  a con- 
tradiction. S 

Select  any  u*  in  U(v^)  and  let  <p ^ be  the  face 

{ (u,v)  eX  : (e,f  (v*))  ((u*,v^)  - (u,v))  -0}  of  X and  let 
^1*  ^ 2 * **•»  ^aces  that  meet  (UCv^J.v*)  . Of 

course,  <p^  contains  U(v^)  x {v^}  . Defining  XA  and  as  in 

Section  5,  we  observe  that  any  point  in  (p^  n (U(v^) , v^)  is  a 


Section  5,  select  6 =»  (min  5^)  > 0 and  6 “ (max  8^)  K 1 
so  that  for  lvQ  - < 6 and  G±(v0)  in  v we  have  *Gi(v(j)  " V*B 

£ 8llv0  - v^S  for  i = 1,  ....  h . The  factor  0 can  be  made 
arbitrarily  small  by  making  <$  sufficiently  small.  If  f'  is 
Lipschitz  continuous  then  there  is  a y so  that 

“W  ■ V*11  - YS  ■ v*"2 

for  sufficiently  small  6 . 

Select  A^  <_  6 so  that  if  (u,v)  is  in  <p^  and  within  a 

distance  A^  of  the  set  U(v^)  x {v^}  then  (u,v)  is  in  the  relative 

interior  of  one  of  the  faces  <p  , ...»  (f h . Select  A^  <.  (1/2)  A^ 

so  that  for  !v  - £ A2  we  have  d(U(v),  U(v*))  _<  (1/2)  A^  . 

Select  e > 0 according  to  Lemma  16  for  d*  =*  (e,f(vJk))  . Select 

A^  £ A^  so  that  for  v^  and  v within  a distance  A^  of  v*  we 

have  HfVg(v)  - f(vJt)N  < e . Select  A^  <_  A^  so  that  for 

Ivq  - v^l  <_  A^  we  have  Ig(vq)  ~ G(v^)l  <_  A^  • 

Now,  for  Ivq  - v^I  <_  A^  we  have  Hg(Vq)  - v^l  <_  A^  . 

Consequently,  lfv0(G(vQ))  - f(vA)l  £e  . Selecting  any  u^  in  U(vQ) 

we  have  (upG(Vg))  in  (p^  . Since  1g(vq)  - v^l  <_  A2  there  is  a 

point  u*  in  UCv^)  with  I(u^,G(Vq))  - (u^.v*)!  <_  A1  . Consequently, 

(u^,G(Vq))  lies  interior  to  one  of  the  faces  <p^,  ...,  (p^  . Therefore, 

G(Vq)  - G^Vq)  for  aome  1 h • Thus,  Ig(vq)  - v^H  <_ 

glvn  - v.l.  Reusing  the  last  expression  we  obtain  G^v-)  -*■  v.  . Since 
0 * U K 


23 


3 can  be  made  arbitrarily  small  by  making  6 sufficiently  small 
we  obtain  Ig^*1^)  - v*l  < BkUGk(vQ)  - V^U  where  &k  ■+■  0 as 
k ® . Similarly,  if  f'  is  Lipschitz  continuous,  we  have 


'GK+V0)  -vj  < YlGk+1(v0)  -vj 


for  k .>  K for  some  K . 

Note  that  if  <p^  = U(v^)  x {v^},  then  Gk(Vg)  = for 
some  k,  that  is,  the  algorithm  terminates. 


Finally,  we  observe  that  there  is  a n such  that 


d(U(v) , U(v*))  < n^v  - v*B 


Thus  we  have 


d(U(G*(v0)),  U(v*))  i»lGk(v0)  - vj 


As  the  figure  illustrates,  the  algorithm  of  the  paper  is  not 


globally  convergent;  let  i m 0,  m - 1,  X ■ [0,5]  . 
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